Good idea! I'll look into that and see if we can add it here. It would make adding more sample binaries easier.

Should that directory just be under /tests? We could maybe try to autoload it unless you specify an alternate location at the command line.

How about tests/binaries or something along these lines? Looking there by default sounds like a great idea, it'll simplify IDE integration. Anything will do as far as I am concerned, no need to wait for another approval.

This second part is bigger than the original change so it probably needs a new review. Most of the time was spent wrangling with the CI actions trying to get the downloading and caching right.

The ultimate goal is to avoid downloading whenever possible, and that part is definitely working: each binary dependency now has its own reusable workflow file with a cache. We now also check the hash for the files just because we can.

actions/cache requires the path parameter to match between the save and restore steps or else it will report a cache miss. My solution to this was to have each sub-action download to the binfiles directory and then add that entire directory to the cache. The main binfiles action restores each the cache for each dependency and they are all merged in the binfiles directory. We then re-cache that using the key binfiles. The end result is that the unit test action only has to restore one cache, not one for each dependency, and not a list of files inside the cache. Adding new sample binaries should be simple because we just need to add a new action and then attach it to the merge action. The unit tests will use whatever is stored under binfiles.

The default location for the binary samples is tests/binfiles. If the files are there and the sha256 hash matches, we will use them in the tests. If not, the tests are skipped by default. If you add the --require-binfile option, the tests fail if we cannot load a dependency. We want this for CI, but new contributors should not see failed tests. You can provide an alternate location for the dependencies with the --binfiles parameter.